ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application

This paper introduces a novel method for translating natural-language instructions into executable robot actions using OpenAI’s ChatGPT in few-shot setting. We propose customizable input prompts that can easily integrate with execution systems or visual recognition programs, adapt to various environments, and create multi-step task plans while mitigating the impact of token limit imposed on ChatGPT. In our approach, receives both textual environmental data, outputs plan an updated environment. These data are reused subsequent planning, thus eliminating extensive record-keeping prior within Experimental results demonstrated effectiveness these across domestic such as manipulations front shelf, fridge, drawer. The conversational capability allows users adjust output via feedback. Additionally, quantitative evaluation VirtualHome showed comparable previous studies. Specifically, 36% planning met executability correctness, rate approached 100% after several rounds Our experiments revealed reasonably tasks estimate post-operation environments without actual experience object manipulation. Despite allure ChatGPT-based robotics, standardized methodology remains elusive, making work substantial contribution. serve templates, offering practical resources robotics research community. source code open publicly available at https://github.com/microsoft/ChatGPT-Robot-Manipulation-Prompts.